Conventionally, in the field of image recognition or speech recognition, a recognition processing algorithm specified for a specific recognition object can generally be performed either using computer software or using a dedicated parallel image management processor (SIMD, MIMD machine and so on).
As an example of the latter approach, i.e., a dedicated parallel management processor, a conventional object recognition device has a plurality of image management processor units, in which an arithmetical operation is performed with DSPs (digital signal processors) mounted on those processor units, and a plurality of results are transferred to an additional unit for making the object recognition (e.g., refer to Japanese Patent Application Laid-Open No. 6-6793). For example, an image is divided into plural areas, which are processed in parallel by the processor units, and the inference for the recognition of the object is made by the additional processor unit, employing the neural network and fuzzy control.
As hardware which performs hierarchical parallel processing using a neural network, for example, a conventional hierarchical structure neural network has an architecture in which a single layer of hardware is used in time division multiplexing and stratified. In other words, a neural network formed by interconnecting a plurality of neuron models includes,
(i) a single layer unit aggregation means which forms a unit aggregate of single layer by installing a plurality of neuron model units that can output a voltage through a non-linear output function in time division, by generating the product of a time division multiplexed analog signal and digital weight data from an outside source and then adding or integrating the product through a capacitor in time division, (ii) feedback means for feeding back the output of the single layer unit aggregation means to an input part of the same unit aggregate of the single layer, and (iii) control means that controls an analog signal from each unit that is outputted by the single layer unit aggregation means in time division multiplexing and controls the single layer unit aggregation means through the feedback means to be employed in time division multiplexing, whereby the neural network having the hierarchical structure is equivalently formed by using the unit aggregation means having a single layer structure in time division multiplexing (e.g., refer to Japanese patent No. 2679730).
On the other hand, a neural network is a mathematical network that is formed based on an information processing system of the brain. Many methods for recognizing and classifying the pattern in an image using a hierarchical neural network have been recently proposed. For example, there is a neural network of the multi-layer perceptron type in which the learning occurs by error back propagation, as is generally widely known. This neural network of the multi-layer perceptron type prepares a number of units equaling the order of the input vector, as an input layer, and the number of categories to be distinguished, as an output layer, and further prepares a sufficient number of middle layers and units, all the units being connected.
For example, a conventional neural network circuit is a multi-layer neural network circuit that recognizes and processes the feature data of a given image by performing the arithmetic operation of the network, in which it is aimed to construct this circuit with small-scale hardware, and to recognize the recognition object of concern at high speed (e.g., refer to Japanese patent No. 03172278). To this end, the speed-up of the operation is effected by performing the operation only for a network having an effective coupling coefficient.
Moreover, another conventional signal processing device consists of a network which combines a plurality of neuronal cell imitation elements with a learning function, each of which inputs plural signals and outputs one signal, in which signal information deletion means for deleting the redundant or unnecessary information of inputs (or outputs) is provided on each of the input side and the output side of the network (e.g., refer to Japanese Patent Application Laid-Open No. 6-83796).
In the image processing requiring an enormous amount of computation as described above, high-speed operation is being realized through parallel processing by special-purpose machines. However, the amount of computation increases as the object to be recognized becomes more complex.
In view of the hardware of the neural network, it is expected that the scale becomes larger to make more complicated processing, considering the number of brain cells of a living body. When the hardware is realized by the conventional method, the number of coupled neurons becomes so enormous that the operation speed of the parallel processing is limited. Also, when the chips mounted on the product are fabricated, it is desired that the size of the circuit and the consumption electric power are made smaller.
In the neural network circuit as disclosed in the above Japanese patent No. 03172278, though the arithmetical operation processing speed is increased by performing the arithmetical operation only for the network of which the coupling coefficient is effective, or not zero, when a noise component is contained in the value of the coupling coefficient, no measurement is taken, whereby the arithmetical operation is finally performed. Moreover, learning of the weight coefficient is not made when the coupling coefficient is 0, because the weight coefficient is updated only for the neurons with a coupling coefficient other than for 0. In other words, universality is expected to be lacking.
Moreover, in the signal processing device as disclosed in Japanese Patent Application Laid-Open No. 6-83796, though the signal information deletion means for deleting the redundant or unnecessary information from the input or output is provided to make the learning easier, the weight coefficient is not learned for the deleted part, as in Japanese patent No. 03172278.